Convolutional Neural Network

Hui Lin

2018-05-21

Neural Network Basic

Input (x) Output (y) Application
Home features Price Real Estate
Ad, user info Click on an Ad? (0/1) Online Advertising
Image Object (1, …, 10) Photo tagging
Image, Radar info Position of other cars Autonomous driving
Audio Text transcript Speech recognition
English Chinese Machine translation
Voice Voice Human computer conversation

Neural Network Example

Computer Vision

Computer Vision

Computer Vision

Image Data

Convolutions Schematic

HTML5 Icon

Edge Detection

Vertical Edge Detection

Parameters

Padding

Strided convolutions

Summary of Convolutions

Convolutions Over Volume

Your Turn: Number of Parameters in One Layer

Question: If you have 10 filters that are \(3 \times 3 \times 3\) in one layer of a neural network, how many parameters does that layer have?

Summary of Notation

If layer \(l\) is a convolution layer:

Pooling Layers

Examples: LeNet - 5

LeCun et al., 1998. Gradient-based learning applied to document recognition

LeCun et al., 1998. Gradient-based learning applied to document recognition

Examples: LeNet - 5

Activation Shape Activation Size # Parameters
Input (32, 32, 1) 1024 0
CONV1 (f=5, s=1) (28, 28, 6) 6272 \(5 \times 5 \times 6 + 6 = 156\)
POOL1 (f=2, s=2) (14, 14, 6) 1176 0
CONV2 (f=5, s=1) (10, 10, 16) 1600 \(5 \times 5 \times 16 + 16 = 416\)
POOL2 (f=2, s=2) (5, 5, 16) 1176 0
FC3 (120, 1) 120 \(400 \times 120 + 1 = 48001\)
FC4 (84, 1) 84 \(120 \times 84 + 1 = 10081\)
Softmax (10, 1) 10 \(84 \times 10 +1 = 841\)

Types of Layer in A Convolutional Network